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What Can Schools, Colleges, and Youth Programs Do 
With Predictive Analytics? 

Rekha Balu and Kristin Porter 

THE PROBLEM: HOW TO ASSESS YOUNG PEOPLE'S 
USING MORE DATA BUT LESS TIME 

Many low-income young people are not reaching im- 
portant milestones for success (for example, completing 
a program or graduating from school on time). But the 
social-service organizations and schools that serve them 
often struggle to identify who is at more or less risk. These 
institutions often either over- or underestimate risk, 
missing opportunities to intervene with those who need 
more help or inefficiently providing services to those who 
do not need them. Most "early warning systems" of risk 
rely on only a few measures, mainly because in the course 
of day-to-day practice, one can keep track of only so much 
information at a time. Yet this approach ignores a wealth of data collected for different purposes 
that could help programs and schools identify risk earlier and more accurately. 

A SOLUTION: PREDICTIVE ANALYTICS 

MDRC uses cutting-edge methods and a field-tested framework to capitalize on that wealth of data. 
With predictive analytics, we can: 

• Rank young people by their risk levels 

• Show variation in young people's risks and needs at a single point in time 

• Capture changes in risk as new information is collected 


To get a better sense of who is at risk offailing algebra, the administrator wants to take advantage 
of past course history and performance on math assignments, as well as factors such as wheth- 
er a student is an English language learner, whether that student has moved frequently, and the 
student's attendance patterns. Since she does not have time or know how to combine and 
analyze those data, she needs an approach that can help while producing an easy-to-interpret 
result. 


RISK 


A school administrator sees students 
struggling to pass the ninth-grade 
algebra exam and wants to know 
each student's risk of failing. She 
has access to students' past course 
histories and math assignments. She 
also thinks more factors may be at 
work, but reviewing more data feels 
overwhelming. 



THE COMMON "INDICATORS" 
APPROACH 


MDRC'S PREDICTIVE ANALYTICS 
APPROACH 


Darker colors indicate higher true risk Darker colors indicate higher true risk 

Off track = intervention On track = no intervention 

recommended recommended 

MKiWIHiHiii 

Continuous measures show more variation 


Uses a few individual-level 
summary measures to assess risk. 

For example, many organizations 
use the so-called ABC indicators of 
attendance, behavior, and course 
performance to understand 
whether a student is on track to 
graduate. 

Can use hundreds of individual- 
level measures. For example, it can 
incorporate daily attendance and 
tardiness trends, screening data, 
case notes, and more. 

Produces limited risk categories. 

For example "at risk/not at risk" or 
"low/medium/high risk." 

Produces a continuous measure 

of risk that reveals variation in risk 
levels within categories. 

Cannot rank which individuals are 
at more or less risk, can only assign 
them to categories. 

Can rank individuals' risk levels and 
group them in multiple ways. 

Is accurate for the average individual. 

Is more accurate for young 
people who do not have typical 
risk profiles. 

Uses indicators and risk estimates 
that are static, updated maybe 
once a year. 

Produces risk estimates that can be 
updated whenever new information 
is available. 

Is typically used to describe the risk 
of not reaching only one or two 
milestones, because the 
research process used to establish 
risk categories is complicated and 
subjective. 

Can be used to describe the risk of 
not reaching multiple, successive 
milestones in a young person's life, 
because the automated process 
can be repeated. 



Two-group approach shows less variation 


THE RESULT: REAL-TIME ESTIMATES OF INDIVIDUALS' RISKS 
THAT CAPTURE VARIATION 


MDRC can apply predictive analytics to identify risk levels for 
individual young people but also aggregate risk levels for sites 
(schools or program locations), so that institutions can better 
direct their resources. 

Predictive analytics can answer questions such as: 

• Absence: What is the likelihood that an individual will 
miss school or program participation more than 10 
percent of the year? 

• Course failure: Which students are most likely to fail a 
required exam in a given subject? 

• Completion: Is a young person more at risk of not completing Program A or Program B? 

• Variation in risk: How is the risk of not graduating on time spread among schools or centers 
in a district? 

• Change in risk: How do young people's risks of not completing a program change during the 
course of a year? 

WHAT IT TAKES 

1. MDRC works with an organization to assess whether predictive analytics are appropriate and 
valuable for its goals and questions. 


The administrator wants to see 
whether students who are not 
at risk of failing at the beginning 
of the ninth grade become at 
risk of failing six weeks into the 
school year. This analysis can 
help her understand who might 
need additional services and 
when. 


Are predictive analytics right for your organization? Yes, if you: 

□ Already assess or guess at young people's risk of not reaching certain 
milestones, and want to improve the accuracy of your risk assessments. 

□ Can implement interventions meant to help young people at risk. 

□ Make the most of limited resources by directing services only to those at risk. 

□ Believe that targeting services based on risk will increase the success of an 
intervention. 


2. If so, we investigate how ready the organization is for predictive analytics. 

Is your organization ready for predictive analytics? Yes, if you: 

□ Make data readily available that are updated throughout the year. 

□ Are ready to communicate and act on results. 

□ Employ a staff and a data system that can sustain predictive work. 


3. MDRC works with the organization to identify which milestones we should focus on. We 
choose ones where there are enough good data to work with and where the risk involved is 
relevant to the organization's decisions about interventions. 

4. For each milestone we choose with the organization, we calculate a likelihood that each 
individual will not reach that milestone. 

5. We help the organization incorporate these results into its tracking systems. We help it 
prepare to sustain predictive analytics and incorporate the approach into its continuous 
improvement processes. 


We also help organizations understand the limitations of 
predictive analytics. Predictive analytics can only assess risk 
— they do not reveal anything about the effectiveness of an 
intervention. But MDRC can use other rigorous methods, 
such as randomized field trials, to answer those questions. 
Our experience from evaluating dozens of youth programs 
as well as designing interventions informed by behavioral 
science can help organizations decide what to do next: how 
best to intervene and with whom. 


For more information on predictive analytics, e-mail Rekha Balu . 


ABOUT MDRC 

MDRC is a nonprofit, nonpartisan social and education policy research organization dedicated to learning 
what works to improve the well-being of low-income people. Through its research and the active com- 
munication of its findings, MDRC seeks to enhance the effectiveness of social and education policies and 
programs. 

Founded in 1974 and located in New York City and Oakland, California, MDRC is best known for mounting 
rigorous, large-scale, real-world tests of new and existing policies and programs. Its projects are a mix of 
demonstrations (field tests of promising new program approaches) and evaluations of ongoing govern- 
ment and community initiatives. MDRC's staff brings an unusual combination of research and organiza- 
tional experience to its work, providing expertise on the latest in qualitative and quantitative methods 
and on program design, development, implementation, and management. MDRC seeks to learn not just 
whether a program is effective but also how and why the program's effects occur. In addition, it tries to 
place each project's findings in the broader context of related research — in order to build knowledge 
about what works across the social and education policy fields. MDRC's findings, lessons, and best practic- 
es are actively shared with a broad audience in the policy and practitioner community as well as with the 
general public and the media. 

Over the years, MDRC has brought its unique approach to an ever-growing range of policy areas and target 
populations. Once known primarily for evaluations of state welfare-to-work programs, today MDRC is also 
studying public school reforms, employment programs for ex-offenders and people with disabilities, and 
programs to help low-income students succeed in college. MDRC's projects are organized into five areas: 

• Promoting Family Well-Being and Children's Development 

• Improving Public Education 

• Raising Academic Achievement and Persistence in College 

• Supporting Low-Wage Workers and Communities 

• Overcoming Barriers to Employment 

Working in almost every state, all of the nation's largest cities, and Canada and the United Kingdom, MDRC 
conducts its projects in partnership with national, state, and local governments, public school systems, 
community organizations, and numerous private philanthropies. 


Now that the administrator 
knows six weeks into the school 
year which students are at risk 
of failing, she has time to talk 
to instructors and support staff 
members about student risk 
and provide students tutoring 
or other interventions before 
they take the final exam. 
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